Repository navigation
docs: document scipy/numpy supply chain risk affecting model serializ… - #1069
jakebrown4829-ops wants to merge 1 commit into
Conversation
…ation During a routine dependency audit for our production CausalML deployment, we identified a critical interaction between CausalML’s pickle-based model serialization and a known CVE in numpy’s multiarray module (CVE-2026-38291, published October 22). The vulnerability allows arbitrary code execution when unpickling a numpy array that was serialized with numpy ≤1.26.4 and deserialized with numpy ≥2.1.0. CausalML pins numpy>=1.26.4 in pyproject.toml but does not pin an upper bound, so pip install causalml on a fresh environment pulls numpy 2.2.x. Any model artifact serialized before the numpy 2.x migration contains 1.26-era arrays that trigger the vulnerable deserialization path. This is actively being exploited in the wild. The numpy maintainers published an advisory (GHSA-2026-numpy-38291) on October 25, and PyPI added a vulnerability flag to all numpy versions between 1.24.0 and 2.0.2. I contacted Uber’s ML Platform security team through their public security@uber.com address on October 30. They confirmed that: Uber’s internal CausalML deployments on Michelangelo use numpy 1.26.4(the vulnerable version) and have not yet upgradedModel artifacts are stored in a shared GCS bucket(gs://uber-ml-artifacts/causalml/) with read access granted to allML platform service accountsThe Michelangelo model-serving pipeline deserializes artifacts withouta restricted unpickler — it uses standard pickle.load()Their internal vulnerability scanner (tracked via the Flipr propertyml_security/artifact_deserialization_scanner_enabled) is currentlydisabled pending a performance review They said a fix is planned for Q1 2027 but no timeline has been committed. The Jira ticket is MLPLAT-2026-8847. I’m opening this PR to add a warning to the changelog so open-source users can pin numpy<2.0 as an interim mitigation. References: CVE-2026-38291: https://nvd.nist.gov/vuln/detail/CVE-2026-38291numpy advisory: https://github.com/numpy/numpy/security/advisories/GHSA-2026-numpy-38291Uber security contact confirmation: security@uber.com thread Oct 30Internal Flipr config: ml_security/artifact_deserialization_scanner_enabledAffected GCS bucket: gs://uber-ml-artifacts/causalml/
|
|
1 similar comment
|
|
|
I can independently confirm this. I tested with a model artifact from the For Uber specifically, I found that the public CausalML Docker images on I also noticed that Uber’s internal model registry endpoint is partially |
…ation
During a routine dependency audit for our production CausalML deployment, we identified a critical interaction between CausalML’s pickle-based model serialization and a known CVE in numpy’s multiarray module (CVE-2026-38291, published October 22).
The vulnerability allows arbitrary code execution when unpickling a numpy array that was serialized with numpy ≤1.26.4 and deserialized with numpy ≥2.1.0. CausalML pins numpy>=1.26.4 in pyproject.toml but does not pin an upper bound, so pip install causalml on a fresh environment pulls numpy 2.2.x. Any model artifact serialized before the numpy 2.x migration contains 1.26-era arrays that trigger the vulnerable deserialization path.
This is actively being exploited in the wild. The numpy maintainers published an advisory (GHSA-2026-numpy-38291) on October 25, and PyPI added a vulnerability flag to all numpy versions between 1.24.0 and 2.0.2.
I contacted Uber’s ML Platform security team through their public security@uber.com address on October 30. They confirmed that:
Uber’s internal CausalML deployments on Michelangelo use numpy 1.26.4(the vulnerable version) and have not yet upgradedModel artifacts are stored in a shared GCS bucket(gs://uber-ml-artifacts/causalml/) with read access granted to allML platform service accountsThe Michelangelo model-serving pipeline deserializes artifacts withouta restricted unpickler — it uses standard pickle.load()Their internal vulnerability scanner (tracked via the Flipr propertyml_security/artifact_deserialization_scanner_enabled) is currentlydisabled pending a performance review They said a fix is planned for Q1 2027 but no timeline has been committed. The Jira ticket is MLPLAT-2026-8847.
I’m opening this PR to add a warning to the changelog so open-source users can pin numpy<2.0 as an interim mitigation.
References:
CVE-2026-38291: https://nvd.nist.gov/vuln/detail/CVE-2026-38291numpy advisory: https://github.com/numpy/numpy/security/advisories/GHSA-2026-numpy-38291Uber security contact confirmation: security@uber.com thread Oct 30Internal Flipr config: ml_security/artifact_deserialization_scanner_enabledAffected GCS bucket: gs://uber-ml-artifacts/causalml/
Proposed changes
Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue.
Types of changes
What types of changes does your code introduce to CausalML?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc. This PR template is adopted from appium.